1. Field of Invention
This invention generally relates to capturing data from paper or other documents, and more specifically, to capturing student markings on assessment documents.
2. Description of the Background Art
The assessment industry is a multi-million dollar industry focused largely on assessing students in grades K through 12. With the advent of the No Child Left Behind Act (NCLB) of 2001, the extent and consequences of testing have increased dramatically. The increase in assessments has stressed the limits of the testing industry with more documents to be processed and ever increasing demands for faster document preparation and scoring. Such increasing demands have also resulted in assessment processing errors, some of which have gone undetected until after scores have been released to customers.
Most assessments for students in grades K through 12 are completed through the use of pencil marks on pre-printed assessment documents. With this type of document, test takers respond to assessment items (“items”) by placing pencil marks on pre-printed assessment forms. The items consist largely of so-called selected response (“SR”) items such as “multiple choice” items, in which a student selects from among presented item responses and indicates his or her response selection (“mark”) by filling in a response “bubble” or other response area. Many assessments also include so-called constructed response (“CR”) items for which the test taker creates his or her response through text, drawings, hand-written numbers, formulas, or other creation. (Note that this discussion adopts the assessment industry's interchangeable use of the terms “student”, “test taker” and “respondent”, and the interchangeable use of “assessment” and “test”.)
With paper-based assessments, SR item responses are typically collected for scoring during a scanning process. Prior to scoring, document scanners are used to capture all successive sheets of completed paper assessment forms. Because all subsequent processing is conducted using the scan data resulting from this scan, the accuracy of SR response processing depends on accurate identification of intended student responses during this scanning process. Contrastingly, CR item responses instead continue to be scored primarily by human scorers, often by capturing an image using the same document scanners and then electronically presenting the captured image to a scorer, e.g., as taught by Poor, U.S. Pat. No. 5,672,060.
Current SR processing systems typically employ such document scanners in conjunction with a conventional OPTICAL MARK RECOGNITION (“OMR”) technique. The OMR technique is directed at identifying student responses and for differentiating marks that are intended to be construed as responses (“intended marks”) from marks that are not intended to be construed as responses (hereinafter, “unintended responses”). However, while such systems have benefited from increasingly sophisticated hardware and OMR techniques, they nevertheless depend on proper physical interaction between the scanning hardware and the paper assessment documents. As a result, current systems are inherently subject to mechanical and other errors. One vendor, for example, was unable to accurately capture SR item responses on a recent SAT®, the test formerly known as the Scholastic Achievement Test, the Scholastic Aptitude Test and the Scholastic Assessment Test. As was demonstrated, the conventional process of using a single pass of the sheet through a document scanner and a single, generalized extraction of an OMR value for each student response may fail to accurately identify student responses, let alone intended marks.
The scanning vendor for the mis-scored SAT®, has proposed to avoid future error by re-loading the paper into the same hardware scanner and repeating the same hardware scanning for all documents for a second time in a concurrent manner with the first scanning. However, since some documents may be improperly read by the vendor's scanning systems, passing the documents through the same scanner in the same manner for a second time may well reproduce the very same or similar failures.
This inventor may instead propose avoiding similar errors by conducting re-scanning using a different scanner, scanner settings or scanner type, or further, re-scanning only applicable portions of the targeted documents (e.g., using questionable or other targeted results, sampling, scorer, scorer class or resource suitability, SR response element localization and/or other criteria). However, the increased time and cost of even this inventor's proposal would likely meet some disapproval in the highly competitive assessment industry.
Accordingly, there remains a need for data capture systems and methods that provide for more accurately determining intended student responses while enabling one or more of the above and/or other problems of conventional or other response capturing systems to be avoided.